Skip to content

dashboard: a slow data run can't hold the page's refreshes - #214

Merged
hhff merged 2 commits into
mainfrom
dashboard-flake-diagnostics
Nov 13, 2026
Merged

hhff merged 2 commits into
mainfrom
dashboard-flake-diagnostics

Conversation

@hhff

@hhff hhff commented Nov 13, 2026

Copy link
Copy Markdown
Member

Fixes the dashboard Chrome tests' flakiness on Node 24: "the note typed" and "1 polls" timeouts in kit (24, rest), which hit #203 three times in four and #204 once. They weren't caused by the code: `dashboard-data` takes about 255 ms on the typing fixture on every branch. The cause was a stall that also affects the owner's real dashboard on a loaded box.

Why it stalled. The page runs one `/api/data` read at a time, and its fetch had no timeout. The server shares one `dashboard-data` child across every page and gives it 120 s. One slow or stuck child stopped every refresh for up to 2 minutes, so each 15 s wait in the tests failed in turn.

The fix

  • Server. A read that has last good data to fall back on waits `DATA_WAIT_MS` (20 s; `DEVTEAM_DATA_WAIT_MS`, where 0 means no wait) on the shared run. After that it answers with the fallback data, marked `x-devteam-stale`. The run carries on and a later read gets its result. The first read, with nothing to fall back on, still waits for the run. An empty output is never kept as good data.
  • Page. Once data is drawn, a refresh gives up after 30 s and says the server didn't answer, so the one-at-a-time guard can't hold. It reads the body before applying the headers, so a read that's given up sets nothing.
  • Press safety, checked in review. A press that reads stale data after writing still shows as done. `ANSWERED` latches until the question is gone from a reply, and overlays skip their time-expiry while `STALE` is set.

Diagnostics. The fixture's fetch hook records each data read. A page wait that times out now reports the last five reads (a stuck one with how long it has been in flight) and the end of the dashboard server's output, so a future failure shows whether the child hung or crawled.

Tests

  • New: "a slow data run…", run with `DEVTEAM_DATA_WAIT_MS=0`. The first read is fresh; a read during a run returns the last good data, marked stale; the run's result arrives on a later read.
  • Node 24 locally: dashboard 113 pass, the rest of the suite 374 pass, bounds 3 pass, none failing.

Review. Fable did an adversarial review. Fixes in the second commit: wait parsing that turned empty or NaN into 0, header state left half-applied after a give-up, empty output treated as good data, the hook comment, and the env scrub. One judgement call is left open: a stale reply has no visual cue on the page beyond the "updated" time. That predates this change; it just happens more often now on a project whose runs take over 20 s.

🤖 Generated with Claude Code

hhff and others added 2 commits November 13, 2026 13:44
A page has one /api/data read at a time, and the server shares one dashboard-data run
across every page for up to 2 minutes. One stuck run on a loaded box stopped every refresh
until it was killed: the Node 24 typing tests' "1 polls" timeouts on CI.

- The server: a read with last good data to fall back on waits DATA_WAIT_MS (20 s) on the
  run, then answers with that data, marked stale. The run goes on; a later read has it.
- The page: a refresh with data already drawn gives up after 30 s and says so, so the
  one-at-a-time guard can't hold. The first read still waits as long as the server does.
- The tests: the fixture's fetch hook records each data read, and a page wait that times
  out reports the last reads (a stuck one with its age) and the end of the server's output.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- DEVTEAM_DATA_WAIT_MS that is empty, negative or not a number is the default, not 0.
- The page reads the body before applying the headers, so a read given up mid-body sets nothing.
- An empty dashboard-data output is never kept as the last good data.
- The test hook's comment says a gate held past 30 s reads as a read given up; the new test
  clears DEVTEAM_SUPERVISOR as its neighbours do.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@hhff

hhff commented Nov 13, 2026

Copy link
Copy Markdown
Member Author

Reviewed: a read with last good data waits at most 20 s on the shared dashboard-data run, then gets that data marked x-devteam-stale. The first read still waits, and empty output is never kept. The page gives up a refresh after 30 s, so load()'s guard can't hold for 2 minutes. Stale data can't cause a wrong action, because owner actions are re-checked by the kit against the live log and the question hash. Fable reviewed it, and its fixes are in. CI 9/9 at 165db33 (kit 24-rest green first time). Follow-up: give stale data a visible cue beyond the "updated" time.

@hhff
hhff merged commit fc2c2d7 into main Nov 13, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant